An image searching system and image searching method, and a 
recording medium storing an image searching program 

BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates generally to an 
image searching system for finding an image that resembles 
a specified image from among images managed by the image 
searching system where the search criteria are determined 
from the specified image. The present invention relates 
more specifically to an image searching system for finding 
an image based on a plurality of specified images. 

Description of the Related Art 

Images stored to a database for managing image 
data are typically stored with additional information that 
can be used to specify search conditions, including search 
keys, such as identifying keywords and image color keys, 
and image features, such as information on the shapes and 
colors appearing in the image. Using this common type of 
image database, it is possible to search the stored images 
to find an image similar to a u ser-specified image 

Image similarity in similar-image searching is 
determined based on such feature quantities as image shape, 
texture, and color. More specifically, the user specifies 
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an image in the image database that is similar to a desired 
image to be searched for. Feature quantities are then 
extracted from the image selected by the user as the search 
key (referred to below as the "key image"), and the 
5 extracted feature quantities are then compared with the 
feature quantities registered in the image database. Image 
similarity indicative of the similarity between the key 
image and a found image is then calculated. Information on 
□ any stored image determined to resemble the key image based 

Hi 10 on this calculated image similarity is then returned as the 
search result . 

j^J Usually there is only one key image selected by 

:L, the user, and it is often impossible to find the one image 

p-. sought by the user based on the feature quantities defined 

-;jfl5 from a single key image. Search accuracy is therefore often 
low. This can happen, for example, when there are plural 
features in a single image and the user focuses on only one 
feature, that is, when the user's search criteria are 
narrower than the criteria that can be extracted from the 
20 key image, as well as when numerous search conditions are 
specified to broaden the range of the search . 

Japanese Patent Laid-Open Publication 7-65169 
teaches one method proposing to solve this problem. 
According to the taught method, a plurality of key images 
25 is specified, a specific area in each of the key images is 



selected, and the features to be used as search criteria 
are extracted from the selected areas. The search is then 
conducted using these features from plural selected areas 
in plural images* This method thus makes it possible to 
search for images having all of the features found in the 
specified areas of the plural key images. In other words, 
this method enables the user to specify a wide range of 
search conditions based on plural images. 

The drawback to said taught method is that the 
user is required to specify in each of the selected key 
images those features that are to be used for searching, 
and this task becomes increasingly difficult and time- 
consuming as the number of key images increases and the 
variety of selected features increases. 

The present invention is therefore directed to an 
image searching system and an image searching method for 
reducing the user burden and improving search accuracy when 
a plurality of key images is specified and a desired image 
is searched for based on these selected key images. 

The present invention is further directed to a 
recording medium for storing an image, searching program 
embodying the image searching method of the present 
invention . 



SUMMARY OF THE INVENTION 
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A first image searching system according to the 
present invention comprises: an image storage means for 
storing a plurality of target images for searching; a 
selection means for selecting a plurality of key images for 
5 determining the search criteria; a first feature quantity 
extraction means for extracting one or a plurality of 
feature quantities from each of the plurality of key images 
selected by the selection means; a second feature quantity 
O extraction means for extracting one or a plurality of 

1*^10 feature quantities from a target image; a feature quantity 
\*\ identification means for comparing a feature quantity 

j*j extracted by the first feature quantity extraction means 

with the same type of feature quantity in each of the 

!l J 

j^l plurality of key images to determine a feature quantity to 

:-J15 be used for searching; a degree of similarity calculating 
means for calculating a degree of similarity between a 
target image and key image using the feature quantity 
identified by the feature quantity identification means for 
searching; and an extraction means for extracting as an 
2 0 image similar to a key image any target image for which the 
degree of similarity calculated by the degree of similarity 
calculating means exceeds a specific value. 

Thus comprised, this first image searching system 
compares features extracted from plural key images to 
25 determine the features to be used as search criteria, and 
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then calculates image similarity using the identified 
features (feature quantities) . 

In this first image searching system, the feature 
quantity identification means preferably defines as a 
5 feature quantity type to be used for searching a feature 
quantity type having an approximately equal value in each 
of the compared key images. 

A second image searching system according to the 
I present invention comprises: an image storage means for 

110 storing a plurality of target images for searching; a 
] selection means for selecting a plurality of key images for 

j determining the search criteria; a first feature quantity 

% extraction means for extracting a feature quantity from 

each of the plurality of key images selected by the 



*1il5 selection means; a second feature quantity extraction means 

■Ms? 

*** for extracting a feature quantity from a target image; a 

first degree of similarity calculating means for 
calculating a first degree of similarity indicative of the 
similarity between a target image and all of the plurality 
20 of key images using the feature quantities extracted by the 
first and second feature quantity extraction means; a 
second degree of similarity calculating means for 
calculating a second degree of similarity indicative of the 
similarity between a target image and at least one of the 
25 plurality of key images using the feature quantities 
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extracted by the first and second feature quantity 
extraction means; a third degree of similarity calculating 
means for calculating a third degree of similarity from the 
first and second degrees of similarity where the first 
degree of similarity is weighted greater than the second 
degree of similarity; and an extraction means for 
extracting as an image similar to a key image any target 
image for which the third degree of similarity calculated 
by the third degree of similarity calculating means exceeds 
J^IO a specific value. The third degree of similarity in this 
*l case is calculated with greater weight assigned to the 

=j first degree of similarity than to the second degree of 

similarity 

A first image searching method according to the 

I; t 

15 present invention searches in a plurality of stored target 
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images for an image that is similar to a key image based on 
the key image. This first image searching method comprises: 
a selection step for selecting a plurality of key images; a 
first feature quantity extraction step for extracting one 
20 or a plurality of feature quantity types from each of the 
plurality of selected key images; a second feature quantity 
extraction step for extracting one or a plurality of 
feature quantity types from the target images; a feature 
quantity identification step for comparing a feature 
25 quantity extracted by the first .feature quantity extraction 
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step with the same type of feature quantity in each of the 
plurality of key images to determine a feature quantity 
type to be used for searching; a degree of similarity 
calculating step for calculating a degree of similarity 
5 between a target image and key image using the feature 
quantity type identified for searching; and an extraction 
step for extracting as an image similar to a key image any 
target image for which the calculated degree of similarity 
13 exceeds a specific value. 

HiO A second image searching method for searching in 

a plurality of stored target images for an image that is 
;;: similar to a key image based on said key image comprises: a 

L_ selection step for selecting a plurality of key images; a 

^ first feature quantity extraction step for extracting a 

*J5 feature quantity from each of the plurality of selected key 

Is? 

^ images; a second feature quantity extraction step for 

extracting a feature quantity from a target image; a first 
degree of similarity calculating step for calculating a 
first degree of similarity indicative of the similarity 

20 between a target image and all of the plurality of key 
images using the feature quantities extracted by the first 
and second feature quantity extraction steps; a second 
degree of similarity calculating means for calculating a 
second degree of similarity indicative of the similarity 

25 between a target image and at least one of the plurality of 
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key images using the feature quantities extracted by the 
first and second feature quantity extraction steps; a third 
degree of similarity calculating step for calculating a 
third degree of similarity from the first and second 
degrees of similarity where the first degree of similarity 
is weighted greater than the second degree of similarity; 
and an extraction step for extracting as an image similar 
to a key image any target image for which the third degree 
of similarity exceeds a specific value, 

A first recording medium according to the present 
invention is a computer-readable recording medium for 
recording an image searching program where the image 
searching program embodies a method for searching in a 
plurality of stored target images for an image that is 
similar to a key image based on said key image. Specific 
steps in this image searching program include: a selection 
step for selecting a plurality of key images; a first 
feature quantity extraction step for extracting one or a 
plurality of feature quantity types from each of the 
plurality of selected key images; a second feature quantity 
extraction step for extracting one or a plurality of 
feature quantity types from the target images; a feature 
quantity identification step for comparing a feature 
quantity extracted by the first feature quantity extraction 
step with the same type of feature quantity in each of the 
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plurality of key images to determine a feature quantity 
type to be used for searching; a degree of similarity 
calculating step for calculating a degree of similarity 
between a target image and key image using the feature 
5 quantity type identified for searching; and an extraction 
step for extracting as an image similar to a key image any 
target image for which the calculated degree of similarity 
exceeds a specific value. 
□ A second recording medium according to the 

iilO present invention is also a computer-readable recording 
!jl medium for recording an image searching program. Specific 

ijrer 

jjl steps in this image searching program include: a selection 

i y 

;^ step for selecting a plurality of key images; a first 

^ feature quantity extraction step for extracting a feature 

- ^45 quantity from each of the plurality of selected key images; 
iy a second feature quantity extraction step for extracting a 

feature quantity from a target image; a first degree of 
similarity calculating step for calculating a first degree 
of similarity indicative of the similarity between a target 
20 image and all of the plurality of key images using the 
feature quantities extracted by the first and second 
feature quantity extraction steps; a second degree of 
similarity calculating means for calculating a second 
degree of similarity indicative of the similarity between a 
25 target image and at least one of the plurality of key 
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images using the feature quantities extracted by the first 
and second feature quantity extraction steps; a second 
degree of similarity calculating means for calculating a 
second degree of similarity indicative of the similarity 
5 between a target image and at least one of the plurality of 
key images using the feature quantities extracted by the 
first and second feature quantity extraction steps; a third 
degree of similarity calculating step for calculating a 
13 third degree of similarity from the first and second 

jlJLO degrees of similarity where the first degree of similarity 

IS 5 

jJl is weighted greater than the second degree of similarity; 

U1 and an extraction step for extracting as an image similar 

^ to a key image any target image for which the third degree 

. |^ of similarity exceeds a specific value. 

113 BRIEF DESCRIPTION OF THE DRAWINGS 

These and other objects and features of the 
present invention will be readily understood from the 
following detailed description taken in conjunction with 
20 preferred embodiments thereof with reference to the 
accompanying drawings, in which like parts are designated 
by like reference numerals and in which: 

Fig, 1 is a typical overview of an image 
searching system according to a preferred embodiment of the 
25 present invention; 



Fig. 2 is a block diagram of primarily the 
control device of the image searching system shown in Fig. 

i; - 

Fig. 3 shows a typical image database table 

structure; 

Fig. 4 is a flow chart of the main loop of the 
image searching method embodied in the image searching 
system shown in Fig. 1; 

Fig. 5 is a flow chart of the image registration 
process shown as step S3 in Fig. 4; 

Fig. 6 is a flow chart of the similar image 
searching process shown as step S4 in Fig. 4; 

Fig. 7 is a flow chart of the search feature 
compilation process shown as step S43 in Fig. 6; 

Fig. 8 is a data table used to describe a method 
of calculating common features; 

Fig. 9 is a data table of sample feature 
quantities extracted from a key image; 

Fig. 10 is a data table of sample feature 
quantities extracted from a target image; 

Fig. 11 is a data table used to describe 
similarity calculation based on common feature quantities; 

Fig. 12 is a flowchart of the similar image 
searching process shown as step S4 in Fig. 4, that is 
executed according to a second preferred embodiment of the 
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present invention; and 

Fig. 13 is a flowchart of the search feature 
compilation process shown as step S43 in Fig. 6, that is 
executed according to the second preferred embodiment of 
5 the present invention. 

Fig. 14 is a data table used to describe 
similarity calculation based on all feature quantities; 

Fig. 15 is a data table used to describe 
13 similarity calculation based on a degree of similarity 

jlJlO based on common feature quantities and a degree of 
similarity based on all feature quantities. 

-.6 ; 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The preferred embodiments of an image searching 
;^15 system according to the present invention are described 
below with reference to the accompanying figures. An image 
searching system according to the present invention 
searches for an image similar to common features found in a 
plurality of specified key images. The image searching 

20 system extracts one or a plurality of feature quantities 
from the plural key images and compares similar feature 
quantities in the key images to determine the types of 
feature quantities to use for searching. The degree of 
similarity is then calculated using the specified feature 

25 quantity types, and an image search is then conducted based 
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on this degree of similarity* It is therefore possible to 
search for images having features common to a plurality of 
key images . 

The image searching system of the present 
invention can alternatively calculate the degree of 
similarity using both a degree of similarity indicative of 
the resemblance to all of the plural key images, and a 
degree of similarity indicative of the resemblance to one 
or some larger subset of the plural key images. This method 

■iJ3 

11J10 makes it possible to expand the range of the search, and 
Ul thus improve search accuracy. 
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Overall configuration of the image searching system 

Fig. 1 shows the basic configuration of an image 
-^15 searching system, referred to as the "system" below, 
according to a preferred embodiment of the present 
invention. As will be known from Fig. 1, this system is 
built around a control device 1 for controlling the overall 
system and comprising a central processing unit (CPU) . 
20 Connected to this exemplary control device 1 are: a display 
2 for displaying text and images, as well as control 
prompts and other interface elements; a keyboard 3 and 
mouse 4 for data entry and operating the system; a floppy 
disk drive 5b in which a floppy disk 5a can be used and a 
25 hard disk drive 6 for reading and writing data to a data 
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storage medium; a printer 7 for printing text and images to 
hard copy; a scanner 8 for capturing image data; a CD-ROM 
drive 9b for accessing data stored to a CD-ROM 9a; a 
speaker 10 for audio output; and a microphone 11 for audio 
input . 

Fig. 2 is a block diagram of the image searching 
system shown in Fig. 1. 

Connected directly to the CPU 201 is a clock 202 
for generating the reference clock required to regulate 



|1J10 system operations. A ROM 203 for storing an application 



program for controlling the image searching system, and a 
RAM 204 for temporarily storing data and programs used by 
the CPU 201 for system control, are connected to the CPU 
201 via a data bus 220. Circuits connected to the CPU 201 
*?15 via the data bus 220 include: a display control circuit 205 
? for controlling the display 2 to display text and graphics; 

a keyboard control circuit 206 for controlling transfer of 
input from the keyboard 3; a mouse control circuit 207 for 
controlling transfer of input from the mouse 4; a floppy 
20 disk drive control circuit 208 for controlling the floppy 
disk drive 5b; a hard disk drive control circuit 209 for 
controlling the hard disk drive 6; a printer control 
circuit 210 for controlling output to the printer 7; a 
scanner control circuit 211 for controlling the scanner 8; 
25 a CD-ROM drive control circuit 212 for controlling the CD- 
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ROM drive 9b; a speaker control circuit 213 for controlling 
the speaker 10; and a microphone control circuit 214 for 
controlling the microphone 11. An expansion slot 215 for 
connecting different types of system expansion boards is 
5 also connected via the data bus 220. 

It should be noted that a SCSI board, for example, 
can be connected to the expansion slot 215 and used for 
connecting a floppy disk drive 5b, hard disk drive 6, 
Q scanner 8, CD-ROM drive 9b, or other device. 

HJ10 It should also be noted that while a floppy disk 

;=] 5a and hard disk drive 6 are the preferred image data 

\i\ storage means in this exemplary system, the invention shall 

^ obviously not be limited thereto as any other data storage 

j^i medium that the system can use can be used for image data 

^15 storage, including magneto-optical (MO) disks [and DVD 
!aV disks] . 

Furthermore, while a scanner 8 is used in this 
exemplary system for image data capture and input, other 
data input devices can also be used, including video still 
20 cameras and digital cameras. 

In addition, while a printer 7 is used as an 
output device, a digital photocopier or other type of hard 
copy output device can also be used. 

A program embodying the control method of the 
25 present invention is also stored to ROM 203 in this 



preferred embodiment. This control program, however, can be 
alternatively stored in whole or in part to another data 
storage medium, including a floppy disk 5a, hard disk drive 
6, or CD-ROM 9a. In this case, the required data and 
program content can be read from the data storage medium as 
needed, buffered to RAM 204, and executed from RAM 204. 

Image database 

To store and manage image data, an image 
searching system according to this preferred embodiment 
also has an image database and a color space table. The 
image database contains both the image data and additional 
information used for the search keys. The color space table 
is referenced when extracting color features from an image. 
Both the image database and color space table are logic 
objects stored on the hard disk drive 6 or other data 
storage medium. 

An exemplary image database 50 in the present 
system is shown in Fig. 3. As will be known from Fig. 3, 
this image database 50 manages the image data to be 
searched, and various columns of image attributes that are 
used as search parameters for finding desired image data. 
These image attributes include keywords, a color key, and 
other search keys, and various feature quantities 
indicative of specific features in the image data. More 
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specifically, these feature quantities include a color 
feature indicative of features relating to image color, a 
shape feature indicative of shapes in the image, and a 
texture feature indicative of texture patterns. The 
5 similarity between images is calculated based on this 
feature information . 



Controlling the image searching system 
p A specific method of controlling the above- 

iljlO described image searching system is described next below 
!i1 with reference to the accompanying flow charts. 



! ; a Main loop of the control program 

is J 

Fig. 4 is a flow chart of the main loop of a 
j^l5 control program run by the CPU 201 in the image searching 
5|s * system shown in Fig. 1. 

The procedure starts by initializing the various 
flags and variables used in the rest of the procedure, and 
to display the default menus (SI) . An exemplary default 
20 menu screen displays various icons representing and used 
for selecting specific processes such that when a user 
selects one of the icons, the corresponding process is 
executed. It should be further noted that the system of 
this preferred embodiment enables the operator to select 
25 various processes and enter information using the keyboard 
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3 and mouse 4 from the default menu screen and other 
screens that are displayed on the display 2. After 
initialization in step SI, decision diamond S2 detects 
whether the user has selected a menu item from the default 
5 menu screen (S2) . 

If in step S2 the user selects "register image, " 
for example, the image registration process (S3) is 
selected and run, and the procedure then advances to step 
S6. This image registration process (S3) is run to store 

10 image data and related feature information in the image 
database 50, and is described in further detail below with 
reference to the flow chart in Fig. 5. 

If in step S2 the user selects "search for 
similar image," for example, the similar image searching 

15 process (S4) is selected and run, and the procedure then 
advances to step S6. This similar image searching process 
(S4) is run to search in the image database 50 for an image 
similar to a selected key image, and is described in 
further detail below with reference to the flow chart in 

20 Fig, 6. 

If in step S2 the user selects another menu item, 
the corresponding process (S5) is selected and run, and the 
procedure then advances to step S6. 

If no menu item is selected in S2, the procedure 
25 advances directly to step S6. After the procedure defined 
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for step S6 is run and completed, the main loop returns to 
step S2 and repeats. 

It should be noted that the tasks that might be 
performed as the "other menu process" selected in S5 are 
5 not unique to or directly related to the present invention, 
and further description thereof is thus omitted below. The 
image registration process (S3) and the similar image 
searching process (S4) of the present invention are 
i;3 described next in detail below. 

njio 

jjl Image registration process 

hi 

Ul The image registration process shown as step S3 

in Fig. 4 is described next in detail below with reference 
to the flow chart in Fig. 5. This process extracts feature 

jij 

Lyi5 quantities from a user-specified image to be stored in the 
image database, and then stores the specified image and 
extracted features to the image database 50. 

This routine starts by reading the user-specified 
image to be stored to the image database 50 (S31) . A 
20 feature extraction process is then run (S32) to extract 
feature quantities from the specified image. In this 
preferred embodiment of the invention, the extracted 
feature quantities include color, shape, and texture, but 
the invention shall obviously not be limited thereto. The 
25 extracted feature quantities are then related* to the image 
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data of the specified image (S33), and finally registered 
in the image database 50 together with the related 
specified image data (S34) . A file name, image size 
information, and other image attributes are also normally 
stored with or related to the image data. The procedure 
then returns to the main loop (step S6 in Fig. 4) . 
[* NOTE: I've used "related" here because the database 
could be a flat file or relational database, and "related" 
will cover the association between feature quantities and 
image data for both database types (the tables in the 
illustrations seem to imply a flat-file (non-relational) 
database structure) .] 

Similar image searching processing 

(a) Similar image search based on common feature 
quantities (first embodiment) 

According to first embodiment, common feature 
quantities are extracted from a plurality of key images. 
Upon evaluating a similarity of a target image to a key 
image, only these common feature quantities are used for 
calculating a degree of similarity of the target image to a 
key image. In other words, feature quantities other than 
the common feature quantities are not used for calculation 
of the degree of similarity. 

The similar image searching processing according 
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to first embodiment (shown as step 4 in Fig. 4) is 
described below with reference to a flow chart shown in Fig. 
6. In the present processing, predetermined feature 
quantities are extracted from each of plural key images 
5 specified by a user and feature quantities indicative of 
features common to the plural key images are selected as 
common feature quantities among the extracted feature 
quantities. Then, the searching processing is performed to 
find out an image similar to a key image among images 

10 registered in the image database 50 based on the common 
feature quantities . 

Referring to Fig. 6, at first, a plurality of key 
images specified by a user are read (step S41) . Feature 
quantities of each key image are extracted from the image 

15 database 50 (step S42). In this step, feature quantities 
of plural kinds such as color, shape, texture and so on are 
obtained from each key image. Next, a search feature 
quantity compilation processing is performed to calculate 
common feature quantities for searching a similar image by 

20 comparing the feature quantities of respective key images 
with each other (step S43) . This search feature quantity 
compilation processing will be described later. 

Thereafter, to find an image among images 
registered in the image database 50 which is similar to the 

25 key images, the common feature quantities calculated in 
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step S43 are compared with feature quantities of each image 
registered in the image database, and the degree of 
similarity between the key images and the presently 
selected image is calculated (from step S44 to step S46) . 
5 Namely, it is determined if there is an image in 

the image database 50 for which the degree of similarity to 
the key images has not yet been calculated (herein below 
referred to as a "target image") (step S44) . If such a 
target image is present in the image database 50, the 

10 feature quantities for that target image are read from the 
image database 50 (step S45) . Then, a degree of similarity 
indicative of a similarity between the target image and 
each key image is calculated based on the common feature 
quantities of the key images and the feature quantities of 

15 the key images and the feature quantities of the target 
image (step S46) and the processing loops back to step S43. 
This calculation processing of the degree of similarity 
will be described later* These steps S43 to S45 are 
repeated until the similarity calculation has been 

20 completed for all target images in the image database. 

When the degree of similarity has been calculated 
for all target images in the image database 50, an output 
processing of search results is performed (step S47) and 
the processing returns to the main routine. In the output 

25 processing of the search results, it is determined that 
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those target images for which the calculated degree of 
similarity exceeds a predetermined threshold value are 
similar to the key images. The target images determined to 
be similar are then displayed on screen in turn from the 
5 highest degree of similarity. 

Alternately, without providing any threshold for 
determination of the similarity, a predetermined number of 
target images can be displayed in a descending order from 
I the highest degree of similarity. Further, the search 

110 results can be displayed by simply outputting the file name 
I of the target image data, preferably with related 

1 attributes such as its location in the database. The 

target images tested by the search loop from step S44 to 
I step S46 shall- also not be limited to all of the images 

^15 stored in the image database 50. For instance, only images 
preselected based on predetermined conditions can be tested. 

The search feature quantity compilation 
processing (step S43 in Fig. 6) is described next with 
reference to a flow chart shown in Fig. 7. This processing 
20 calculates the common feature quantities to be used upon 
searching a target image which is similar in common to a 
plurality of key images. That is, the common feature 
quantities indicative of features in common to a plurality 
of key images are obtained by comparing feature quantities 
25 of the same kind among the plurality of key images and 
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choosing feature quantities of the same kind having values 
being approximate to each other as the common feature 
quantities. This processing is described in further detail 
below . 

5 As shown in Fig. 7, the processing starts by 

calculating, for every kind of the feature quantity, 
differences among the feature quantities of the key images 
(step S431) . Next, a mean value of the differences 
ri obtained in step S431 is calculated for every kind of the 

Hi 10 feature quantity (S432) . An example of this calculation is 

Ul 

Ul shown in Fig. 8. This figure shows an example obtained for 

Us? 

Ul three key images 1 to 3 each having five feature quantities 

;=„ 1 to 5 as shown in Fig. 9. The mean value of the 

differences with respect to the same feature quantity 
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^15 indicates the smaller the mean value the greater the image 



similarity with respect to that feature quantity. 

Thereafter, every feature quantity or the kind of 
the feature quantity having a mean value equal to or less 
than a predetermined threshold is selected as a common 

20 feature quantity (step S433) . That is, when the mean 
feature quantity difference is less than the threshold 
value, that feature quantity is determined to be common to 
the plural key images. In the example shown in Fig. 8, 
this threshold value is set to 0.1 and feature quantity 2 

25 and feature quantity 3 are thus selected as common feature 
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quantities. Thus, the feature quantities or kinds of 
feature quantities to be used for searching are determined 
by comparing feature quantities of the same kind in the key 
images. In the last step, a specific flag is set for every 
5 feature quantity selected as a common feature quantity 
(step S434) . 

NeSfcst^, the processing for calculating the degree 
of similarity (\tep S46 in Fig. 6) is described below. 
Based on the commorN^feature quantities extracted from the 
njlO key images and a target image, this processing calculates 



the degree of similarity^ of the target image to all key 
jjl images or either one of chenK 

According to the present embodiment, only feature 
jlj quantities selected as common feature quantities in step 

UJ15 S43 are used for calculating the degree of similarity of 
*J3 the target image to the key image. 

Accordingly, feature quantities other than common 
feature quantities £sre not used for the calculation of the 
degree of similarity. \Due to this, images having features 
20 common to plural key images can be obtained as the results 
of search with a high accuracy. In this case, the target 
image similar to all of the plural key images. In other 
words, this search method imposes a search under an AND 
condition among the key images anck thereby, enables to 
25 restrict ranges of searching conditions \o reasonable ones. 



( iN^- 11 shows an example of the calculation for 

the degree of \similarity . In this example, the distance 
between each key\jriage and the target image .is calculated 
based on the valuek of feature quantity 2 and feature 
quantity 3 selected asVthe common feature quantities and 
the degree of similarity is obtained based on the 
calculated distances. The\ values shown in Fig. 11 are 
obtained from the following equations (1) to (3) . 

Difference between feature\ quantities = | (key image 
feature quantity) - (target image feature quantity) | (1) 

Distance = square root of Ngum of (each feature 
quantity difference ) 2 \ (2) 

Degree of similarity = 1 . O/distancey (3) 

The degree of similarity thus obtained indicates 
that the image similarity becomes high as the calculated 
value of the degree of similarity becomes large, 
(b) Similar image searching based on all kinds of feature 
quantities (second embodiment) 

According to second embodiment/ the degree of 
similarity between target image and key images is 

calculated for each of\the key images based on feature 
quantities of all kinds. \From the degrees of similarity 
calculated for each key im^e, the highest degree of 
similarity is then selected as th^e degree of similarity of 
the target image to the corresponding key image. In this 
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case, images similar to at least one of the plural key 
images are searched. That is, the image search is 
performed under an OR logic, condition and, accordingly, it 
becomes possible to broaden the ranges of searching 
5 conditions by increasing the number of key images. Second 
embodiment performs a processing shown in Fig. 12 and Fig. 
13 instead of Figs. 6 and 7 of first embodiment. 
J*>>V J> X eferrin 9 to Fi 9- 12, a plurality of key images 
specified by\a user are read at first (S141) . Feature 

10 quantities are obtained from the image database 50 for each 
key image (S142 ). \. Namely, feature quantities of plural 
kinds such as color\ shape, texture and the like are 
obtained form each key in^ge . 

Then, the feature quantities of each key image 

15 obtained in step S142 are compared with feature quantities 
of each target image registered in the image database 50, 
the similarity between each key image and the target image 
is determined and thereby, target images being similar to 
respective key images are searched (steps from S143 to 

20 S145) . 

^A^^p At firs1\ it is determined whether or not there 

is a target image rn the image database for which the 
degree of similarity t>o each key image has not yet 
calculated (step S143) . If\such a target image is present 
25 in the image database 50, the\feature quantities for that 
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target im^ge are read from the image database (step S144) . 
Then, calculation of the degree of similarity indicative of 
similarity between images is performed based on the feature 
quantities of each key image and those of the target image 
5 (step S144) . Afterv this calculation, the processing loops 
back to step S143. \This calculation processing of the 
degree of similarity (s\ep S145) will be described later. 
These steps S143 to S145 a^e repeated until the similarity 
p calculation has been completed, for all target images in the 

jijlO image database 50, 

Ul 

Ul When the degree of similarity has been calculated 

in for all target images in the image database 50, an output 

processing of search results is performed (step S147) and 
TU then, the processing returns to the main routine. In this 

WIS output processing of search results (step S146) , based on 
U3 the degrees of similarity calculated for all target images 

in the image database 50, the key image having a degree of 
similarity higher than a predetermined threshold value is 
determined to be similar to the corresponding key image. 
20 The target images determined to be similar are then display 
on screen in sequence from the highest degree of similarity. 

Alternately, without providing any threshold for 
determination of the similarity, a predetermined number of 
target images can be displayed in a descending order from 
25 the highest degree of similarity. Further, the search 



results can be displayed by simply outputting the file name 
of the target image data, preferably with related 
attributes such as its location in the database. The 
target images tested by the search loop from step S144 to 
step S146 shall also not be limited to all of the images 
stored in the image database 50. For instance, only images 
preselected based on predetermined conditions can be tested. 

Next, the calculation processing of degree of 
similarity is described below with reference to a flow 
chart shown in Fig. 13. 
^J>j£- As si 

quantity of the key\image and that of the same kind of the 
target image is calculated for each of all feature 
quantities, and a distance between the key images and the 
target image is calculated based on the distances of 
respective feature quantities 

Based on the distance between images, the degree 
of similarity is calculated (step S1451) . More 
specifically, the values shown in Fig. 14 are obtained from 
the following equations (4) to (6). 

Feature quantity distance = | (key image feature 
quantity) - (target image feature quantity) | (4) 

Distance = sqrt of {sum of (each feature quantity 
distance) 2 } (5) 

Degree of similarity = 1.0/distance 
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Note that with this method image similarity 
increases as the calculated degree of similarity value 
rises. Because the degree of similarity to key image 3 is 
highest in the example shown in Fig. 14, that value is used 
5 as the degree of similarity between the current target 
image and the key images (step S1452) . 

The output processing of search results mentioned 
above with respect to step S14 6 is performed based on the 
p degrees of similarity determined in step S1452. 

jljlO (c) Similar image searching processing using degrees 

U1 

ijl of similarity based on all feature quantities and degrees 

.Ijl of similarity based on common feature quantities (third 

i: embodiment) 

HJ This method combines the degrees of similarity 

jti 

UJ15 based on the common feature quantities calculated according 
*J3 to first embodiment (a) and those based on all feature 

quantities calculated according to second embodiment (b) , 
to obtain final degrees of similarity. In this case, the 
degree of similarity based on common feature quantities is 
20 weighted and the weighted degree of similarity and the 
degree of similarity based on all feature quantities are 
added to obtain a mean value thereof. This mean value is 
assigned as the degree of similarity of the target image to 
the key image. Thus, this method performs a similar image 
25 search in combination of AND and OR conditions. As a 
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result, it becomes possible to find an image having 
features common to all key images among images similar to 
any one of the key images . and therefore, to improve the 
search accuracy compared with second embodiment (b) . 



5- \£n example of the calculation of the degree of 



^ similarity ^ss shown in Fig. 15. In this example, the 



weight for theNiegree of similarity a calculated in Fig. 11 
is set larger than that for the degree of similarity 
calculated in Fig. V 14. A weighted mean value is than 

10 calculated to obtain xthe final degree of similarity c. 

This final degree of similarity c is obtained according to 
the next equation (7) 

Degree of similarity = { (degree of similarity a x 
weight wa)+ (degree of similarity b * weight wb) } / (wa+wb) 

15 (7) 
^^^P^ A ^ stated above, according to the present 

embodiment, feature quantities are extracted from a 
plurality of key rs^iages, when specified and those common to 
the plural key imaged are selected to determine similarity 

20 between images using the^e common feature quantities. Thus, 
the common feature quantities are automatically determined 
from among plural key imagesNand accordingly, the user 
burden can be reduced. It is alscK possible to search for 
images having features common to plural key images. This 

25 enables to narrow the ranges of resp^qtive searching 
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conditions by specifying plural key images. Also, the 
search cark be similarly expanded to broader ranges of 
searching conditions by . considering the degree of 
similarity indicating similarity of an image to at least 
5 one of plural key images. Search accuracy can be thus 
improved. 

As will be known from the above descriptions of 
the present invention, a first image searching system, a 
first image searching method, and a first recording medium 

10 storing an image searching program embodying the first 
image searching method of the invention determines the 
feature quantities to be used for image searching based on 
the feature quantities extracted from one or more key 
images, and calculates a degree of similarity using only 

15 the feature quantities selected as search criteria. It is 
therefore possible to search for images with features 
common to those of the selected key images. The search 
conditions can thus be restricted by specifying plural key 
images, and search accuracy can thus be improved. The user 

20 burden is also reduced because feature quantities are 
automatically extracted from the specified key images, and 
similar images are searched for based on the extracted 
feature quantities . 

As will be known from the above descriptions of 

25 the present invention, a second image searching system, a 
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second image searching method, and a second recording 
medium storing an image searching program embodying the 
second image searching method of the invention calculates 
the degree of similarity used for searching by combining a 
5 degree of similarity indicative of similarity to all of the 
plurality of key image, and a degree of similarity 
indicative of similarity to any one of the plurality of key 
image. It is therefore possible to set a wide range of 
search criteria by selecting a plurality of key images, and 
HJIO further improve the accuracy of similar image searches . 

I 

U 3 

Ul Although the present invention has been described 

j{! in connection with the preferred embodiments thereof with 

t '5T 

u _ reference to the accompanying drawings, it is to be noted 

|^ that various changes and modifications will be apparent to 

Hil5 those skilled in the art. Such changes and modifications 
AjJ are to be understood as included within the scope of the 

present invention as defined by the appended claims, unless 

they depart therefrom. 



